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ABSTRACT 

Many applications of robots require that the same task be repeated a number of times. In 
such applications, the errors associated with one cycle are also repeated every cycle of the 
operation. An off-line learning control scheme is used here to modifythe command 
function which would result in smaller errors in the next operation. The learning scheme is 
based on a knowledge of the errors and error rates associated with each cycle. Necessary 
conditions for the iterative scheme to converge to zero errors are derived analytically 
considering a second order servosystem model. Computer simulations show that the errors 
are reduced at a faster rate if the error rate is included in the iteration scheme, pie results 
also indicate that the scheme may increase the magnitude of errors if the rate information is 
not included in the iteration scheme. Modification of the command input using a phase and 
gain adjustment is also proposed to reduce the eirors with one attempt. The scheme is then 
applied to a computer model of a robot system similar to PUMA 560. Improved 
performance of the robot is shown by considering various cases of trajectory tracing. 1 he 
scheme is also applied to a real robot PUMA 560. The results show that the proposed 
scheme can be successfully used to improve the performance of actual robots within the 
limitations of the repeatability and noise characteristics of the robot. 
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1.0 INTRODUCTION 


Several methods of performance improvement of robots have been attempted previously by 
many researchers. Some of these employ on-line adaptive control schemes 1 ’^ considering 
factors such as flexibility of the arms and variation of loads. Such control schemes are 
required for trajectories which are not defined a priori or require accurate trajectory control 
in the first attempt. For repetitive type of operations, as most commonly required in 
industrial applications, such as welding, cutting or sealing, a control scheme which can 
learn based on its previous performance appears to be more attractive because of the 
simplicity of the technique. A method to obtain a modified command input signal called 
Computed Repetitive Adjustment Technique (CREATE)3 is embodied in an algorithm for 
correcting a robot s motion in successive testing of the same job. This repetitive testing is 
continued until the trajectory errors are within the acceptable bound before performing the 

actual work. The technique was later applied 4 ’ 5 to improve the performance of a 
mathematical model of a three-link robot arm. 


A scheme very similar to CRE. 
linearized model of the robot 
minimum errors were obtained 

system are known. Anmoto"^ proposed a learning scheme based on measuring error rate 
only. He later extended®’" the control scheme to include the error and error rate 
information. However, the conditions 8 he arrived at are found to be unsatisfactory. 

Several other researchers 10 ’ 1 M2 have attempted to obtain convergence conditions for the 
iteration scheme, but the analysis are generally inadequate. Togai 1 ^ 14 obtained interesting 
algorithms by using discrete analysis and by applying optimal control techniques. An 
algorithm based on optimal control technique is also given by Harokopos 15 for continuous 
systems. Bedewi 16 used CREATE technique to refine the performance of the robot 
starting with a dynamic inverse of the model of the system. In this paper, the learning 
control scheme proposed by Arimoto 8 will be considered. Analysis of the scheme will be 
attempted with a servosystem model. The scheme will then be applied to a mathematical 
model of a three-link robot to show the improvement in performance, 


ATE was studied by Craig^ for application by considering a 
Conditions for the convergence of the scheme to yield 
with the assumption that several critical narameterc nf th,» 


2.0 SECOND ORDER SERVOSYSTEM MODEL 

2.1 Description of the Servosystem Model 

A servo control design is given in this section which will be applied to design independent 
control of each joint of the robot. The second order servosystem model is also used in this 
section to test the proposed learning control scheme. The scheme is then applied to 
improve the performance of the robot model in section 3. 

'Die dynamics of the servosystem can be represented, by ignoring the damping factor for 
the sake of simplicity, as 

T C (I ) 

9 is the angular position (a function of time, t) 


10 = 
where 


Tc is the control torque 


I is the moment of inertia of the servo, and 
0 is the second derivative of 0 with respect to time 


It is necessary to feedback the angular position, 0 , and its rate, 0 , to obtain a stable 
position control system. An integral feedback of the position may also be used to obtain a 
higher speed of response. However, an integral feedback increases the order of the system 
and introduces oscillations in the system response. These characteristics are not desirable 
for the present application and hence only a proportional plus derivative controller is 
considered for stabilizing the servosystem as shown in Fig. 1. The characteristic equation 
of the closedloop system can be written as, 

S 2 + («2/i)S + (K i /i) = 0 (2) 

This equation may be compared with the damped oscillatory system 

s 2 + 2£co n s + = 0 

where £ is the damping ratio and co n is the natural frequency of the system. It is desirable 

to have a high value of co n to get a large bandwidth of the system. The stability of the 
second order system is guaranteed as long as Ki and K2 remain positive. However, the 
feedback gain value Ki can not be increased beyond a certain a limit (due to the practical 
limitations of the actuator) in an effort to increase the bandwidth of the system. Within 
these limits, the feedback gain values are obtained using the relations (obtained from Eqs. 
(2) and (3)) as, 

Kj = to n 2 i and K 2 = 2£a> n i 

The rate feedback gain, K2, is assumed to achieve critical damping in the closedloop 
system. For such a case, the system will have the highest speed of response corresponding 
to the lowest settling time and for a given bandwidth of the system. Fig. 2 shows the 
transient response of the second order system for a step input and for the selected feedback 
gain values. 

Case (a) = 100 K 2 = 20 (o n = 10 rad/sec 

(b) Kq = 144 K 2 = 24 (On = 12 rad/sec 

(c) Ki = 625 K 2 = 50 co n = 25 rad/sec 

Case (c) shows the lowest rise time of approximately 0.2 sec. The same feedback gain 
values will be considered later to control the robot arm joints. 


2.2 A Learning Control Scheme 

The second order system considered may be represented as (from Fig. 2), 

(is 2 + K 2 S + K | ) 0 = 0c 


1, the command function, 0 C , is the same as the desired trajectory, 0 C , is 


0d - 0 a ) + b(0 d - 0 a ) ( 6) 

re constants proportional to the error rate and the error, respectively. For 
i the command signal, 0 C , is modified as, 

! c + 0 e (7) 

the system to this modified command signal is determined. The new error 
obtained using Eq. (6) The command signal is again modified according to 
V command signal is used in Eq. (2.5) and the response of the system is 
: process is repeated until the errors, 0 e , fall within an acceptable value. 

idy is possible for the second order system considered here. Eq. (5) 
second order system is written in a slightly different form for the kth 


0 k = 0 ck 

(8) 

:eration the equation as 

1 + K,0 k+ , = 0 ck + 0 ek 

(9) 

k ) + b(0<j - 0 k ) 

(10) 

:8) from Eq. (9), we get 

= 0 ek 

(11) 


'k 


ad k + bd k 


( 12 ) 


(t) dr 


4 


( 13 ) 


c>' .-op: ».* 
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Then 


t 

Jk + 1 = j 8 2 ek+lW dT 
0 

= j I” 0 ek - (ad k + bd k )J dx 

o L 


Expanding and using equations (13) and (10), we get 

t 

Jk - J k + 1 = j {(2aq - a 2 - 2b) d k 2 + (2bp - b 2 ) d k 2 } dx+ ad k 2 (0 

0 

+ (ap + bq -ab) d k 2 + 2bdkdk 

i 0 , for a 2 > 2b 
a < 3q/2 

b < 2p and for small error rates (14) 


Eq.(14) gives sufficient conditions for the guaranteed convergence of the error to zero 
value with each step of the proposed iteration scheme. 

Since only the command signals are being modified, the torque requirements on the 
actuators will not drastically change provided the initial errors are small. The feedback 
values selected for the application in this section are Ki = 100 and K 2 = 20 and 
corresponds to a closedloop frequency of 10 rad/sec. The response of the servosystem to a 
command input, 

0(3 = sin (tot) 

is shown in Fig. 3 for 0 ) = n rad/sec. The actual trajectory shows a phase lag and an 
amplitude modification. Application of the learning control scheme shows reduced errors 
with the first iteration (Fig. 4) and the third iteration (Fig. 5) and for the parameters a- 0. 1 
and b= 0.9. The figures compare average errors associated with each iteration. The average 


as 


r.r; .. 

Of PC’-'.,'* ~ f ,.y 


AE 


NP 

2 I 0, 
1 = 1 


ei 


NP 


( 15 ) 


mber of points on the trajectory. Higher values of to were considered next 
lance of the iteration scheme was studied. Fig. 6 shows the average error 

each iteration for various values of "a" and "b" and for co = 2 tc rad/sec. 
rly show that for the case in which a = 0, the errors tend to increase after a 
n accordance with the derived conditions (Eq. (14)). Similar results are 

for co =4 n rad/sec. In general, the results show that the rate of 
.Teases with increasing values of "a" up to a certain value. For the present 
best results are obtained with a= 0.2 and b= 0.8 at all values of co. The 
ing error rate in the modification procedure is obvious from Figs. 6 and 7. 
bserved that the conditions given by Eq. (14) are rather conservative 


1 Phase Adjustment Technique 

he previous results that there is a unique command function, 9r , for 

1 trajectory, 9 a , will be the same as the desired trajectory, at least for 
! single-output systems. In the previous examples, inclusion of the error 
helped a great deal in arriving at the unique command function with only a 
rhe next obvious question is whether it is possible to obtain the ideal 
on with a single iteration so that the resulting trajectory would follow the 
y. An attempt has been made in this section to answer the question. 

of the second order system shows that the output of the system follows the 
ly, provided the frequencies associated with the function are well within the 
th. As the input frequency approaches the system bandwidth, the output of 
s marked deviation from the input signal. TTie input-output relationship is 
se die gain plots in which the phase difference and the ratio of the 
; output to the input are plotted as a function of the input frequency. 

the previous example, 9(j = sin(cot), with co = n rad/sec. The time 
>ystem is as shown in Fig. 3., for the first trial. A modification scheme 
was employed with a=0. 1 and b=0.9. The actual trajectories obtained 
)n and third iteration are shown in Figs. 4 and 5, respectively. From Fig 3 
ues of phase (-0.5969 rad) and gain (0.91) were obtained. The command 
dified as 

1 sin(cot + 0.5969) 

tory obtained for this modified command signal is shown in Fig. 8. The 
and the desired trajectory overlap within the accuracy of the plotter. Thus, 
linear system, it was possible to obtain a modified command signal to 
ed trajectory without an iteration scheme. This method will be applied to 
del of three-link robot system in section 3, which is an example of a 
lie system. 
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3.0 APPLICATION TO A THREE-LINK ROBOT 

3.1 Dynamics of a Three-Link Robot Arm 

Fig. 9 shows the selected coordinate reference frame for the three-link robot arm. X, Y, Z, 
is the inertial system with the origin at joint 1. The fourth coordinate frame has the origin 
fixed at the tip of the robot arm. The first link can rotate only about the vertical axis Z, 
carrying the second and the third links. The second link can rotate about an axis fixed in 
link 2 and is normal to both the first and the second link. The third link moves about an 
axis parallel to the axis of rotation of the second link. 

The three transformations are represented by the following matrices 


X 


Cl 

-Si 

0 

0 


Xi 

Y 

_ 

S 1 

Cl 

0 

0 


v 1 

Z 


0 

0 

1 

dl 


Zi 

1 


0 

0 

0 

1 


1 

or 

r = [ Ai J 

Ti 




Xi 


1 

0 

0 

0 


X 2 

Vi 

s 

0 

C 2 

-S 2 

C 2 d 2 


V 2 

Zi 


0 

s 2 

C 2 

S 2 d 2 


Z 2 

1 


0 

0 

0 

1 


1 

or 

Ti = 

[ A 2 ] r 2 




*2 


1 

0 

0 

0 


x 3 

V 2 

= 

0 

C 3 

-S 3 

C 3 d 3 


v 3 

z 2 


0 

S 3 

C 3 

S3d3 


z 3 

1 


0 

0 

0 

1 


1 


(16) 


(17) 


or r 2 = I A3] f3 


(18) 


where Cj= cos 0j, and Sj = sin 0j 5 j= 1,2,3 

A point rj described with respect to link i can be related to the base coordinate by 

r = [ Ti ] ri ( 19 ) 

where, Ti = [ Ai ], T 2 = [ Ai ] [ A 2 ], and T 3 = [ T2 ] [ A 3 ] 

The dynamic equation for the robot arm is obtained as, ^ 
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(20) 


3 .. .. 33 

Fi = 2 D, j qj + q'i + 2 2 D^kqjqk + Dj 



j-1 


j»i 

k-l 

where: 




°ij 

11 

M w 

Trace 

( |IE Jp 

3T p t ) 

1 J 

P ■ 

max i , j 

dqj 

dq ( 

Dijk 

3 

= 2 

Trace 

( J2T P j 

P ££> 

P 3 

; max i J 

dqjdq^ 

dq ( 

Di = 

3 

2 - 
i = 1 

■ m p g T 

dT n T 
-r-1— r p 

dq, p 



I a j is the I ^ 1 actuator inertia, mp is the mass of the p ^ 1 link 
g is a vector in the direction of the gravity force 
Jp is a pseudo inertia matrix for the pth link 

Eq. (20) are used to compute the joint torques required. For the purpose of simulation of 
the first, second and the third links are assumed to have lengths equal to 0 . 5 m, 0 . 4 m, and 
masses 4kg, and 2 kg, and 1kg, respectively. The inertias of the links 1,2, and, 3 are 
assumed to be 5xl0‘ 4 ’ and 2.5xl0' 4 > and 1.25X10' 4 kgm 2 , respectively. Actuator inertias 

corresponding to joints 1 , 2 , and, 3 are taken to be lxlO' 3 , 5xl0‘ 3 , and 2. 5x1 O' 4 , kgm 2 
respectively. 

3.2 Inverse Kinematics 

The trajectory to be traced by the robot arm is given by a set of points which are usually 
defined in the base (inertial) coordinate system. Therefore, it is necessary to find the joint 
angles corresponding to each point on the trajectory using the geometry of the robot. Let 
us assume that the tip of the robot arm follows the given trajectory, so that a transformation 
of the tip point into the base coordinates can be made as follows: 


(P) = [T3] (r 3 ) (2i ) 

where r 3 is actually the origin of the fourth coordinate system. Substituting for T 3 and 
expanding we get, 

P x = -S](C2D2 + C23D3) ( 22 ) 

Py = C 1 (C2D2 + C23D3) ( 23 ) 

P z = D 1 + D2S2 + D3C23 ( 24 ) 

Algebraic manipulations of the above equations give the expressions for the joint angles as 

9, = -tan-’ (xj/y,) (25) 
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cos'M 


J2 + d 2 2 - d 3 2 

2dd 2 


1 /C 1 ) 2 + (Zi -d])2 - d 2 2 - d 3 2 , 
2d 2 d 3 


(27) 


ampute the joint angles only once in order to find the command signals 
ry. This can be done off-line using the robot central computer in the 
iterative technique is then applied to modify the joint angle commands to 
jle errors to zero. 

rol system 

• employed here is to control the motion of the joints so that the tip of the 
trajectory. Independent control of each of the joints based on only the 
position and rate feedbacks is considered here for the sake of simplicity, 
led in the base coordinate system is converted into joint command angles 
kinematics. The dynamic equations (20) are rearranged for the 
ol techniques as follows. 

3 . . 3 

I D ijk 0j0 k - 2 Dij0j - Di> / (Di, + 18*) (28) 

k= 1 j=1 

j*1 

int angular coordinates, Tci is the torque about the joint i. Control 
ed from the block diagram shown in Fig.l as 


< i (0i “ 9 C ) (29) 

a sixth order nonlinear differential equation with coupled coefficients, 
numerically solved here using a fourth order Runge-Kutta integration 


of Learning Control Scheme 

the case in which the desired trajectory for the tip of the arm is a circle, 
ined in the inertial coordinate system as follows. 


s(cot) 

).5 sin (cot) 


(30) 


am is modified so that the joint command angles are obtained from the 
/stem through a kinematic transformation. The computer time required 
approximately 20 minutes and hence only four iterations are considered 
rhe results obtained are shown in Fig. 1 1. There are two curves in 
ling to the errors associated with X and Z coordinates. The errors in 
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the Y direction is not so important and is not shown here. It may be seen that the errors 
tend to become smaller at a faster rate when the error rate information is included in the 
command input modification procedure. Fig. 12 shows the torque characteristics of the 
joints 1 and 2 before and after the application of the learning control scheme. Figs. 13 to 
15 show the actual response of the robot corresponding to a=0 and b=l. A trajectory very 
close to the desired circular trajectory could be obtained with a=0. 1 and b=l after 6 
iterations (figure not shown). 

3.5 Application of Gain and Phase Adjustment Technique 

It was shown in section 2.3 that a modified command input could be obtained so that the 
resulting actual trajectory is very close to the desired trajectory for linear dynamic systems. 
However, the robots are in general are characterized by nonlinearities and varying moments 
of inertia parameters. Coupling between the joint coordinates are also very significant. 

Two examples of trajectory tracing, one a circle and the other a straight line, is considered 
for the application of the gain and phase modification technique. 

The trajectory defined by Eq.(30) describes a circle in the X-Z plane. The actual trajectory 
traced by the robot is given in Fig. 13. The responses of the robot after first and fourth 
iterations using the learning scheme are as shown in Figs. 14 and 15, respectively. It may 
be seen that even after four iterations the trajectory traced is still not a complete circle. The 
gain and phase modification technique was then attempted as follows. 

The response of the system to the command trajectory which is the same as the desired 
trajectory was plotted in the X and Z coordinate system as a function of time (not shown). 
Ideally, X coordinate should be a cosine function and Z a sine function as given by Eq. 
(30). However, depending on the value of w, there will be a phase and gain change. The 
actual response was found to fit the following functions. 

X = C0S(o>t - 0.17) (31) 

Z = sin(cot - 0. 1 9) 

Modified command signals were then obtained as 

X = cos(cot + 0. 1 7) (32) 

Z = sfn(ot + 0. 1 9) 

The response of the robot to the modified command signal is as shown in Fig. 16. The 
actual trajectory obtained is seen to be very close to the desired trajectory. It is very 
interesting to see that the nonlinearity, the coupling between the joint coordinates, gravity 
and the varying moments of inertias did not have much effect as far as the phase shift and 
gain values are considered. The present example, however, is very simple because the 
desired trajectory could be represented by simple sine and cosine functions. 

The second example considered is the one in which the tip of the robot traces a straight line. 
The joint angles will have to go through a nonlinear motion so that the tip of the robot could 
result in a straight line. Hence, this example is more complicated than the first case. The 
command function in the inertial coordinate system for the example of a straight line is 
taken to be 


X * 0.5 - 0.5 w t 
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The resDo* system with w = 1 is as shown in Fig. 17. For the sake of simplicity, 
only theXte^is considered for modification. Learning control scheme is then used 
to obtain arajcctory after 6 iterations as shown. For the application of gain an 
phase mod, the command signal for the X coordinate is obtained as fol o 

X =0.5 wt + X e (t + tp)Xg 

where X e ior obtained as 

*e x a 

and t p is thcshift obtained from the first trail. The response obtained with this 
modified coi is shown in Fig. 17. The phase t p is taken to be 0.18 secs and the 
gain, X g , 1. trial and error). The actual trajectory obtained for this command is 
almost as gohe trajectory obtained through 6 iterations. 

The phase, t.gain, X e , were also obtained by a Fourier analysis of the error function 

associated w trajectory. Fig. 18 shows the response of the co ^^ d “f e ^ s fied 
command (w= 0 .2 secs and X g = 1.117). The response appears to be at least as 

good as the rse shown in Fig. 17. Hence, it may be concluded that die Founer 
analysis can Id to compute the phase and gain value to obtain a modified 
and to reducerrors considerably in the second trial. Further minimization of errors 
may be done ; the iterative technique, if necessary. 

3.6 Applicalto a Real Robot 

The perfonnarmorovement technique is attempted on a real robot PUMA 560. Fig 19 
shows a schem diagram of the robot. Instructions to the robot arm is given y u g 
operating systeoftwarc, VAL-H 18 . Different values of the coefficient of the error are 
considered (wi (he parameter ’a’=0) to study the rate ofconvergenceof the 
yield minimumrorv Fig. 20 shows the performance of the robot after 1 \^^ons fo 
desired trajectorof the robot to move along the Y axis at approximately 1 ^ TTje 
figure shows therrotin the X direction with the first attempt and after! 1 UCTanonswith 
b=0.2. The erros associated with the first tnal is 0.4145mm and , after 11 iteracnons , 
0.07 mm. 

The same case vws then repeated with various values of the error coefficient, b, and th® 
average errors obtained as a function of the trial number are shown m Fig. 21. ne results 
clearly show thatthe rate at which the average error decreases is P ro ^ m ™ ^ * e ri ^ e 
b. However, larger values of b tends to increase the average error after a few iteratio . 
This trend was also observed and discussed in section 2. 

The iterative technique was also applied to improve the performance of the t robot -^d®* ; * 
variety of other operating conditions. The results show, in general, that * . 

the average error after reached (which is 0.1mm) is restricted by the repeat ab ty 
noise associated with the joint sensors. Further work is continuing at tins dme 
the rate information in the application of the learning scheme to the real rooo 
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4.0 CONCLUSIONS 


An off-line learning control scheme is analyzed here which can be used to improve the 
performance of robots. Necessary conditions for the iterative scheme to reduce errors with 
each iteration are derived considering a second order servo system model. The scheme is 
also applied to a mathematical model of a three-link robot similar to PUMA 560. In 
general, the results show that the learning control scheme reduces errors more efficiently if 
the error rate information is included in the scheme. The results also show that the scheme 
may increase the magnitude of the errors, if the rate information is not included in the 
iteration scheme. Preliminary results of the application of the technique to a real robot has 
shown that the scheme can be successfully used to improve the performance of actual 
robots within the limitations of the repeatability and noise characteristics of the robots. 
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Fig. 18. Response of the Robot for a Modified Command Input 
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